POLICY BRIEF 


Trends in Third Grade 
Reading Proficiency 


An Analysis of DC CAS Results (2007-2014) 


Executive Summary 

The early elementary grades are a critical time 
for students to develop the academic and 
developmental building blocks they will need 
to succeed in school. In particular, learning to 
read proficiently by the end of third grade is 
one of the best predictors of a student's later 
academic success. 


Our analysis of DC CAS scores found that third 
grade reading proficiency did not improve for 
students citywide between 2007 and 2014. 
Additionally, we found a statistically significant 
decline in reading scores for economically 
disadvantaged and black third graders. We 
also found that economically disadvantaged 
third graders attending schools with high 
concentrations of poverty scored significantly 
worse on the reading assessment than those 


attending more economically integrated schools. 


Given the results of this analysis and the body 
of research demonstrating the importance 

of mastering reading by the end of the third 
grade, District leaders must reexamine how 
resources are allocated and prioritized starting 
at birth. To fully support literacy development, 
we must work to ensure the city’s early 
childhood and elementary programs are both 
comprehensive and evidenced-based so that 
all children can develop the skills they need to 
succeed in school. 


A growing body of research tells us that 
achieving proficiency in reading by the end of 
third grade is one of the best predictors of a 


student's later academic success. This 
message has not been lost on educators 

and policymakers; as states and school 
districts seek to identify critical educational 
benchmarks, third grade proficiency has 
become the target of a broad range of policy 
interventions across the country. 


In 2012, DC Action for Children released 

a policy brief examining the District of 
Columbia's progress in raising third grade math 
and reading proficiency. The brief found that 
despite a wide array of sweeping educational 
reforms under the DC Public Education 
Reform Amendment Act (PERAA) and the 
implementation of several policies intended 
to improve early reading ability, there was no 
statistically significant change to third grade 
math and reading proficiency between 2007 
and 2011.1 This report updates our analysis 
to include the final years the DC CAS was 
administered in DC;' providing both a richer 
data set and a more comprehensive look at 
how third grade students fared under PERAA. 


From Learning to Read to Reading to 
Learn: Why Third Grade Proficiency 
Matters 

The third grade represents an important 
juncture both in the way children learn and 
the way they are taught. During the first eight 
years of life, the human brain experiences its 
most rapid growth and is most malleable to 
environmental influences. Since this cognitive 
development occurs cumulatively, with later 
growth building on top of existing abilities, the 


' For the 2014/2015 school year, the DC CAS was replaced by the Partnership for Assessment of Readiness for College and 
Careers (PARCC), a standardized assessment based on the Common Core standards that is being adopted in a number of 
school districts across the country. Because of the differences in psychometric properties between these two assessments, 
it will not be possible to compare the new PARCC results with old DC CAS results. 
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first eight years of learning are critical 
for future success.’ Similarly, early 
elementary grades focus on developing 
foundational knowledge in literacy, 
math, and reasoning that are essential for 
success when students encounter more 
complex content later in their schooling. 
As a result, it is critical for students to 
develop the academic building blocks of 
core subject material by the time they 
finish the third grade. 


The importance of the third grade as 

a benchmark for success is more than 
just theory or conjecture: a large-scale, 
longitudinal study released in 2012 by the 
Annie E. Casey Foundation found that 
students who do not read proficiently by 
the end of the third grade are four times 
less likely to graduate high school. These 
effects are compounded for children in 
impoverished families and those living in 
neighborhoods with high concentrations 
of poverty— two findings that carry 
significant weight for many children in 
the District. Failing to graduate from high 
school can have serious consequences 
for one’s life outcomes, limiting 
employment opportunities, lowering 
earnings and increasing the likelihood of 
poor health and criminal activity. 


Data and Limitations 

This policy brief analyzes school-level 
DC CAS results, which are reported on 

a four-tiered scale: Below Basic, Basic, 
Proficient and Advanced. While DC CAS 
results are often expressed in terms of the 
percent of students scoring proficient or 
above—an easy way to gauge a school's 
performance at a glance—this method 
would not capture an improvement in 

a student's score from one year to the 
next, such as an increase from Below 
Basic to Basic or Proficient to Advanced. 
To get a more nuanced picture of the 
data, we use a weighted proficiency 
formula," which assigns each proficiency 
level a value between 1 and 4: a weighted 
proficiency score of 1 would mean 

all students scored Below Basic and a 
weighted proficiency score of 4 would 
mean all students scored Advanced. 

In this update, two types of statistical 


analysis were used to determine whether 
trends in reading proficiency scores 
between 2007 and 2014 were statistically 
significant. Since third grade reading 
proficiency, in particular, is associated 
with improved outcomes, this update will 
focus on trends in reading performance, 
looking particularly at how race/ethnicity, 
socioeconomic status, and school sector 
are associated with student success. 


There are two primary limitations for this 
analysis. First, since publicly available 
DC CAS results are reported at the 
school level, the depth of the analysis 
possible is inherently restricted. For 
example, we know that the District is 
undergoing significant demographic 
changes, but without student level 

data it is not possible to control for the 
differences in student characteristics 
between different third grade cohorts 

or between student subgroups. Second, 
while the weighted proficiency formula is 
an improvement over an analysis of the 
percent of students scoring proficient, 

it is not able to capture the fine-grained 
changes between the upper and lower 
bounds of each of the tiered proficiency 
levels. In an ideal world, access to the 
raw test results, at the student level, 
would allow for a more thorough analysis 
and understanding of the results. For 


3rd Grade Reading Proficiency: 
Students Citywide (2007-2014) 


more information about the data and 
methodologies used in this analysis, 
please see the appendix. 


Results: Third Grade Reading 
Proficiency in the District Today 
Third grade reading proficiency in the 
District has not improved between 2007 
and 2014. In fact, our analysis indicates 
that there was actually a slight downward 
trend in reading proficiency between 2007 
and 2014. However, while this change was 
statistically significant, it was small enough 
that it was not substantively meaningful. 
Our update confirms our finding from 

the original 2012 publication: the reading 
proficiency of third graders citywide has 
not improved since the passage of PERAA. 


Economic Disadvantage and 
Proficiency 

In the District, the poverty rate for public 
school students is high: 66 percent of 

all DCPS and public charter students are 
eligible for free and reduced price meals 
(FARMs), a common proxy measure 

for student poverty. A closer look at DC 
CAS results sheds additional light on the 
role that poverty plays in constraining a 
student's academic potential. In 2014, only 
34 percent of economically disadvantaged 
third-grade students tested proficient or 
above in reading. By contrast, nearly 78 


3rd Grade Reading Proficiency: Economically 
Disadvantaged Students (2007-2014) 
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" The weighted proficiency formula was created for our original third grade analysis in partnership with the Elder Research Group, who assisted with the quantitative analysis. 
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Our analysis of weighted 
reading proficiency scores 
suggests a sobering trend: 
the proficiency rates of 
black third-grade students 
have actually declined over 
these years, while they have 


risen for white students. 


percent of the students not designated 
as economically disadvantaged scored 
proficient or above on the same test. This 
gap provides a striking illustration of how 
the unequal distribution of resources can 
shape a child’s educational opportunities. 
While this gap is significant, recent 
changes in the way DCPS calculates 

and reports FARMs eligibility suggest 

that this gap may be even larger." Our 
analysis of DC CAS results between 2007 
and 2014 found a statistically significant 
downward trend in reading proficiency 
for economically disadvantaged third 
grade students. 


Racial and socioeconomic segregation 
in the District over many years has led to 
highly concentrated poverty in many of 
the city’s neighborhoods. For example, 
nine of the ten neighborhood clusters 
with the highest poverty rates are east 
of the Anacostia River in Wards 7 and 

8.4 This socioeconomic segregation is 
especially prevalent in the city’s public 
schools, which can further compound 
the disadvantages faced by students 
living in poverty at home. Schools with 

a high percentage of the student body 
living in poverty have higher teacher 
turnover rates and less qualified teachers, 
on average, than schools with a larger 
percentage of middle or upper-class 
students.° Research suggests that a 
school’s socioeconomic makeup may 
have as much impact on students’ 
academic growth over time as their 
individual economic status® and that 
low-income students attending low- 
poverty schools perform substantially 
better academically than those in schools 
with high concentrations of poverty.’ 
Our analysis of third grade reading 
proficiency in the District supports these 
findings. We found that economically 
disadvantaged students attending 
schools with lower concentrations of 
economically disadvantaged students 
outperformed those in high-poverty 
schools. On average, economically 


disadvantaged third graders who attend 
a DCPS or PCS school where 25 percent 
of the student body is economically 
disadvantaged scored more than one 
standard deviation higher on their 

third grade reading assessment than 
economically disadvantaged students 
who attended schools where all students 
were economically disadvantaged — a 
score higher than the citywide average 
for all students. 


Race/Ethnicity Proficiency 

While race/ethnicity itself does not 
influence academic achievement, it is 
deeply connected with socioeconomic 
status and opportunity in the District, 
where the racial achievement gaps 

are some of the widest in the country. 
Indeed, while 94 percent of white third 
grade students tested proficient or above 
in reading in 2014, the same was true for 
only 35 percent of black third graders and 
36 percent of Hispanic third graders. Our 
analysis of weighted reading proficiency 
scores suggests a sobering trend: the 
proficiency rates of black third-grade 
students have actually declined over 


3rd Grade Reading Proficiency: 
By Race/Ethnicity (2007-2014) 
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"In 2013, DCPS began participating in a federal program that allows schools to extend eligibility for free and reduced meals to all students at high poverty or ‘high 
need’ schools without determining the eligibility of individual students. While this measure reduces the administrative burden on schools, and helps ensure that more 
low-income students have access to meals at schools, an undetermined number of higher income students are now classified as ‘economically disadvantaged’ as a 
result. This change in the way schools classify economically disadvantaged students has likely inflated the average DC CAS results in 2013 and 2014 for economically 


disadvantaged students as a whole. 
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these years, while they have risen for 
white students. While Hispanic proficiency 
scores appeared to decline, our analysis 
did not find statistically significant 
evidence of this trend. Taken together, 
these results indicate that the racial literacy 
gap between black and white third graders 
in the District may have actually increased 
over the last eight years. 


School Sector 

There is much debate around whether 
attending a charter or traditional public 
school leads to different academic 
outcomes for students.” One common 
finding in much of the rigorous research 
on the subject has been that a wide 
range of school quality can be found 

in both charters and traditional public 
schools.® This holds true in the District, 
as schools performing well above, and 
below, the city’s average can be found in 
both DC public schools and DC public 
charter schools. In 2014, both sectors 
had a similar reading proficiency rate: 
43 percent for DCPS and 45 percent 

for charter schools. However, our 
analysis of weighted reading proficiency 
scores indicates a statistically significant 
downward trend in reading proficiency 
for DCPS students but no trend for 
charter students. 


A closer look at the data suggests that 
economically disadvantaged third graders 
in charter schools are performing better 
at reading than those in DC public 


3rd Grade Reading Proficiency by Sector 
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schools. Forty-one percent of third 
graders in public charter schools tested 
proficient in reading in 2014, compared 


to only 29 percent of DCPS third graders. 


Our analysis of DC CAS results indicates 
a downward trend in reading proficiency 
for economically disadvantaged DCPS 
third grade students between 2007 and 
2014, but we failed to find a statistically 
significant trend in reading proficiency 
for economically disadvantaged charter 
school students over the same time. 
We are unable to determine whether 
the difference in the performance of 
economically disadvantaged students 
between school sectors is the result of 


differences in the quality of education 
provided or other factors. 


Moving Forward 

Today, more than half of the city’s third 
graders are still not proficient in reading, 
making it clear that we have not made 
the necessary progress to give the city's 
young students the tools they need to 
succeed in school and later in life. For 
those who work to improve the lives and 
outcomes of the District's children— 
whether in the school system, the city 
government, or nonprofit sector— the 
lack of significant gains in reading 
proficiency should be cause for serious 
reflection. While living in or near poverty 
is undeniably a major barrier to academic 
achievement for many of the District's 
students, we can, and must do better. 
The city invests significant resources in 
its universal pre-kindergarten program, 
has adopted a series of accountability 
reforms and continues to have the 
highest per-pupil expenses in the country. 
However, as past research indicates, it 

is not necessarily how much is spent 

on education, but how it is spent. 

We must reexamine both how we 
allocate resources, from birth through 
graduation, to ensure we prioritize the 
evidenced-based programs that most 
benefit the District's students, and that 
social programs that deal with the 
effects of poverty are integrated with 
the city’s schools. 


“It is often difficult to untangle the effects of schools from selection bias; students who attend charter schools may differ in a number of ways from those who attend 
public schools that may not be readily observable, like parental involvement and student motivation. 
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Appendix I: Methodology 


This publication is an update toa 

2012 report undertaken by the Elder 
Research Group on behalf of DC Action 
for Children. In order to maintain 
comparability, this update attempts to 
replicate the methodological approach 
undertaken by the authors of the original 
report as much possible. 


Data 

This analysis employs data from the 

DC CAS test, provided by the Office of 
the State Superintendent of Education 
(OSSE). The test results are summarized 
at the school level and are reported on 

a four-tiered scale: Below Basic, Basic, 
Proficient, and Advanced. School-level 
results are further disaggregated by 
student subgroup including racial/ethnic 
categories and low-income status. 
Since the publication of the original 
report, OSSE has begun suppressing DC 
CAS results for school-level subgroups 
with less than ten students. In order to 
maintain consistency, the school-level 
results of student subgroups with less 
than ten students were removed from the 
analysis for the previous years. 


For this update, we explore the 
differences in mean, weighted reading 
proficiency scores for all third grade 
public school students between the years 
2007 and 2014 using linear regression 
and ANOVA tests. A weighting system was 
introduced to the data to provide more 
information in progress shifts over time 

in the four respective result categories. 
Every student's test score is weighted 

on the following scale: Below Basic-1, 
Basic-2, Proficient-3, and Advanced-4. 


A school with students scoring all 
Proficient and none Advanced would 
receive a 3.0 where schools receiving all 
Advanced scores would receive a 4.0. 


Analytic Approach 

We explored changes in reading 
proficiency rates using least squared 
regressions and an Analysis of Variance 
(ANOVA). 


After ordering the school level results 

by subgroup, a least squared error 
regression line was fitted over weighted 
scores for the eight years included in the 
analysis. The slope of the line is tested 
using a t-distribution to determine if the 
slope is different than zero. A p-value for 
each fitted line is subsequently reported 
to show the level of significance that the 
slope has. A slope with a p-value greater 
than .05 would indicate that there has 
not been a statistically significant change 
in average test scores over the eight- 
year period. Because there were only 5 
years of data available for the original 
report, the authors determined that this 
approach had some inherent weaknesses 
by violating basic assumptions of the 
underlying approach. 


As a back-up to this approach, ANOVA 
was used to test for the differences in 
data over time with a higher level of 
confidence using the year as a classifier 
and comparing the means of each 

year with every other year with the null 
hypothesis being that all years are equal. 
For this update, school-level weighted 
test scores for each subgroup by year, 
were compared in the ANOVA. 


With ANOVA, a difference in just one year 
will give this test a significant result. If an 
ANOVA test is significant, in order to avoid 
a Type 1 statistical error, it is necessary to 
run a post hoc test to see how the years 
compare to each other. Since the original 
report did not clarify which post-hoc test 
it used, the Ryan Procedure (REGWQ) 

was employed in this analysis. REGWQ is 
a post hoc test for multiple comparisons 
that strikes a balance between the Tukey 
test's firm control over familywise errors 
and the statistical power of the Neuman 
Keul's test, thereby representing a balance 
between the risk of Type | and Type II 
statistical errors. It also accommodates the 
analyses with many pairwise comparisons. 


Limitations: 

The data provided has been summarized 
at the school level, which limits the 
questions that can be answered and 
reduces the level of confidence in the 
answers provided. For example, it was 
not possible to control for compositional 
changes in third grade cohorts, or for 
student demographic characteristics. 
Since student test scores are reported 
using the four-tiered scale, they are also 
not suited for capturing fine-grained 
changes between the upper and lower 
bounds of each proficiency level. The 
data and collection methods are not 
suitable to allow inferences about causes 
for the levels of or changes in proficiency. 
For future work, reporting the raw test 
results, at the student level, would 

allow for a more thorough analysis and 
understanding of the results. 


DC Action for Children * 1432 K Street, NW, Suite 1050, Washington, DC 20005 » 202-234-9404 « @ActforDChildren » info@dckids.org *» www.dckids.org 


